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ABSTRACT 

This paper suggests ways in which counselors can take 
greater control of their accountability plans and make evaluations 
serve both "the counselor and the school more constructively. 
Recommended competencies for specific types of guidance programs are 
described in detail. Three types of evaluation data are discussed for 
use in accountability models, i.e., outcome, opinion, and enumerative 
data. An 'accountability model that blends the data together is 
suggested as an effective strategy for providing a broad range of 
information about results of individual and program efforts, student 
attitudes, and cost-effectiveness. Samples of data collection 
instruments are included in the appendices. (JAC) 
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Accountability for School Counseling Programs 

Evaluation is not a new idea* However, I suspect that most school 
counseling programs do not have a formal evaluation system established. 
There are a number of possible reasons for this ranging from personal dis- 
interest in the idea to lack of time for such activities. In my opinion, 
it is unrealistic for school counselors to think that they need be accountable 
ouly to themselves. I also believe that it is unwise for counselors to 
passively allow others to impose evaluation plans upon them because such 
plans may be unfair to counselors. For example, such plans may emphasize 
purely objective outcomes causing counselors to have to do more of everything 
in order to look good. In addition, non-counselors, when given the 
responsibility to develop evaluation plans, may not be aware of important 
things to evaluate. 

Background 

School guidance is^what one writer has called the "third force. 11 The 
first and most established force in the schools is the teachers who are 
primarily responsible for the cognitive domain. The second force, which is 
also the second most well established, is the administrators whose primary 
concern is with maintaining an orderly atmosphere, for education. Counselors, 
the third force, are the newest professionals on the scene, and they are not 
-as~well~established~in the "schools as are teachers and administrators. 
Counselors, of course, are primarily concerned with the affective needs of 
students. 

A 

Fueled by the National Defense Education Act of 1958 and the influx of 
the post World War II baby boom children upon the overcrowded schools, 
school counseling flourished in the 1960's- It was a period of rapid growth 



and confidence. Perhaps the high water mark was signalted in 1970 when 
Dugald Arbuckle 1 asked the question:* "Does the School Really Need Counselors. 1 
He supported this question by citing the following conditions: 

1. The wide gap between preparation and practice in school jcP.uu.seling.-_ 

2. The questionable uniqueness -of the counselors 1 services. 

s 

3. Evidence that there is little or no research data supporting the 
effectiveness of counseling. 

4. Apparent disinterest in professional affiliations or professional 
organizations demonstrateci~by a number of counselors. 

A review of the professional literature since 1970 will reveal" numer(!%s 
references- to justification, survival, and accountability. 

As this nation has become increasingly aware of expenditures and desirous 
for cost-effectiveness during the economic difficulties of the past decade, 

counselors, as well as others, are being ^caj-led upon to be accountable. 

• , - * . \ 

Consequently, *we can safely say that counseling too has entered r into the 

age of accountability. 

Solutions 

As was stated earlier, I believe that it is unwise for counselors to 
passively allow others to impose evaluation plans upon them. Consequently , 
we need to take control of our own accountability plans as much as' possible 
and^make them constructively serve us. as well as the institutions for which 
we work. With the preceding philosophical position in mind, I would like 
to share some of my ideas with you. 




l Dugaid S. Arbuckle. "Does the School Really Need Counselors," 
School Counselor,^ 1970, 17, 325-330. 



Recommended Competencies 

Counselors who desire to competently present evic^nce of accountability 
for others and for their own improvement and development should be able to: 

1. Conduct useful-needs— assessments. 

2. Translate ideas into measurable goals and objectives. 

3. State measurable goals and objectives. fi 

4. Systematically plan and carry out goals and objectives. 

5. Collect relevant data. , 1 

6. Construct useful data collection instruments (surveys, questionnaires, 
etc.). 

7.. Accurately tabulate numerical data. 

8. Keep accurate records of desired information. 

9. Generate cooperation and motivation among involved guidance staff 
members.^ 

10. Determine which data are^ relevant. * a 

11. Understand the meanings .associated with acquired- accountability data. 

12. Clearly share the results of their accountability efforts with others. 
I have listed the competencies which are stated above in order to provide 

food Lor thought as you read through the following portions' of this paper. 
Types of Guidance Programs 

^ Perhaps the most desirous _type of guidance_pxogram^is one which is founded 
on appropriately stated objectives upon which an evaluation plan can be 
developed in ortter to determine whether or not the objectives are being 
attained. A more common type of guidance program either has no objectives or 
else the stated objectives are too broad, vague, and general to lend themselves 
to an evaluation of outcomes. 
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Types of Evaluation Data 

In ray opinion, there are three kinds of useful data for evaluating 
guidance programs. Each adds a dimension which -the others do not provide, 
and together they provide a comprehensive evaluation system which has the 
potential for a useful accountability model. 

Outcome data. Outcome measures provide a means of evaluating programs 
with clearly stated objectives. An example follows: 

Objective: Help students to learn how to make decisions wisely. 

Related Activity: Initiated a group counseling program which 
included vocational simulation and outside exploratory 
assignments. \ r * 

< Outcomes: Nineteen s'cudents mastered an applied eight-step 
vocational decision-making process, ending with a written 
tentative action plan; each verbalized how the process could 
apply to a new decision; six verbalized the process but did 
not apply it to their own vocational 'decision. 2 

Readers can see that the stated outcomes report how well the goal was 

met through the particular activity that was employed. Clearly, outcome 

data take the form of reporting what happened as a result of activities 

which were established to carry out objectives. „In an objectives-based 

program, the developmental sequence will be to establish objectives, determine 

activities, and collect outcome data. In a program without objectives, 

° f 1 

outcome data are inappropriate unless somehow one is aKIe tojtevelop 
objectives for well established activities and then collect the necessary 
data. In this instance, the developmental sequence is to determine useful 
activities, develop appropriate measurable objectives, and collect outcome 
data. Many existing guidance programs may have to do this in order to be 



2 John D. Krurnboltz," "An Accountability Model for Counselors, "Personnel 
and Guidance Journal, 52 (June, 1?74), 639-646. 



able to have outcome data' as part of their accountability program. More 
about this later. 

Opinion data . Opinion surveys and questionnaires provide a means for 
acquiring information about the subjective attitudes of various guidance 
consumers, e.g., students, parents, teachers, graduates, administrators. 
Opinion data are usually acquired via some form of questionnaire or survey 
which presents questions or statements that can be answered with ratings, 

words, or statements. Several examples follow: 

\ 

Directions: Circle the response that best describes the services 
provided by your counselor. 

My counselor tries to be helpful Yes No 

or 

Seldom Always 
My counselor tries to be helpful 1 2 3-4 5*6-7 

In a program with objectives, outcome data can be supplemented by opinion 
data because, while it is useful to know how whether objectives are being met, 
it is also important to know about the attitudes of participants. It is 
possible, for instance, to successfully achieve goals while concurrently 
dissatisfying one*s consumers. 

Where programs do not have objectives, opinion data are one of the 
legitimate forms of evaluation because stated objectives art* not absolutely 
necessary for the development of questions focusing upon consumer opinions. 
In creating opinion surveys, it is very important to develop survey items 

7 

which a6k the right questions in the right way. If not, evaluators run the 
risk, of collecting inappropriate data, neglecting to collect important infor- 
mation, confusing respondents, or failing to assess opinions on all Oi the 
important components of their program. I*t would appear that careful pLanning 
should accompany the development of any opinion surveys *hich are to be used 
to evaluate guidance programs. 
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Enumerat i ve * Pat a . This appr&ach takes the form of tallying how many 
jLimes^.or-dur-ing- what -duration^ -some event* or behavior took place. This 



information, in turn^ may .be compared with the amount of time available 

» * 

and/or cost per units. These kinds of data are the easiest to collect and 
also are the kind most frequently collected. Examples of t^he numerous 

counseling behaviors -that can be enumerated or tabulated are: .number of 

• • - »• 

counseling sessions, types of counseling, sessions, number of parent contacts 

• * « 

length of interviews, time spent per function, totaj. time spent in providing 
• • • • * 1 

services, cost in dollars #per total services and/or specific services, * 

* 

services requested, services performed versus services requested, aj\d time 

spent on non-guidance services. A couple of examples follow: 

The survey form used in the recent PDOE study of activities of 

Pennsylvania school counselors is an example of enumerative data collection. 

See Appendix A. The form allows counselors to check cells indicating whom 

* • • 

they were serving, how much tithe yas> spent, what concerns were involved, and 

what methods we. . used. In another survey which was conducted by the Erie, 

• • • . 

Pennsylvania school district, color coded cards were used. An example is 

/ . • • 

found in -Appendix B. An interesting combination of enumerative and outcome 

data is found in the ideas of John D. Krumboltz .who was cited earlier. See 

Appendix C. . 

As was the case with opinion data, enumerative data can also supplement 

• • • 

outcome data because, as seen in the Krumboltz example, tfost^ef fecfciveness 
is an important supplement to information about achievement of objectives. 
Where programs do not have useful objectives, enumerative data, like opinion 
data, are also one of the legitimate forms pf evaluation because objectives 
are not absolutely necessary to determine enumerative data. 
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The warning about overemphasis upon objective information to the point 
of undue emphasis upon quantitative dataTand measurement of effectiveness 
according Co the most of or least of something, e.g., most clients seen or 
least amount of money spent per student, is particularly important when 
dealing. withenumerative data. Counselors heed to be aware of the potential 
for damage involved with undue emphasis upon the quanitif ication involved in 
enumerating things. This .is why enumerative data alone are not as useful 
as they are when combined with other types of data. 

* Combining Different Kinds of Evaluation Data Into An Effective 

Accountability Plan 

In my opinion, the ideal data collecting plan for a counseling program 1 t 
accountability model is one which combines outcome, opinion, and enumerative 
data. When blended together, they provide a broad range of complimentary 
information about results of individual aod program efforts, attitudes of j 
consumers, and cost-effectiveness of one's activities. 

If a guidance program is already founded upon well developed objectives 
the^B will be. little or no problem acquiring outcome data. However, where 
such objectives do not exist, two alternative suggestions are offered: 

1. As was suggested earlier, one can start with accepted program 
activities and move backward to stating related objectives. For example: 
In a hypothetical program, all counselors made an effort to get well 
acquainted with their advisees during their first year in^school because 
they think its a good idea. However, they never stopped to think about 
the intended outcomes. After giving this some thought, it was decided that 
one purpose or objective was to increase the number of independent self- 
referrals to the counselors by students at a later data. The get-acquainted 
interviews were viewed as a means of increasing the number of independent 
sellf-ref errals. This, of course, is a measurable objective. 
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2. Start all over, so to speak, by conducting a needs assessment and 
using the results as a. foundation for setting program goals and objectives. 
Of course, many exist ing_program activities will probably be supported by 
the newly established objectives. I think the best way to conduct needs- * 
assessments is to assess the needs of the counseling staff, the students, 
the administration and staff, and the public (taxpayers) and to diplomati- 
cally derive gome compromise objectives fox the guidance program. ^/Initially,* 
this wili involve quite a bit of work. However, after having established 
sound objectives, counselors may find that their future accountability 
activities' are much easier and that their consumers have greater respect for 
their work knowing what the counselors 1 intentions are. 

There are ways through which outcome data can be acquired systematically, 
with" efiix^iency and ease, and in a manner which provides information that 
is also useful for purposes other than accountability demands. One approach 
is the method demonstrated in the Krumboltz plan. Another method which ties 
in, with tracking the progress of counseling cases, is the single case study 
in that it provides graphic evidence of individual progress on mutually 
determined counseling objectives. Appendix D proyi^es--e^camples of type A 
and AB designs, which are^ easy--t^rreco^ paper and, not. only 

.serve as evidence for accountability purposes, but also as evidence of client 
progress in counseling. 

Of course, the Krumboltz plan also provides a procedure for collecting 
\gnumerative data in that the amount of counselor time spent per activity is 
•recorded and translated into cost per hour. Another interesting plan for 

collecting enumerative data is thp one used in the Erie public school's 

t, * 

several years ago. This, plan offers a streamlined procedure for collecting 
enumerative data. The Erie staff developed several different color coded 
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cards, about the same size as those used in ^ata* processing, 9 Each card 1 
contains a t£tle whifch, is keyeck.to its colour, e«g«, Individual Student 
Counseling (red), Individual Student Information/Service '(blue)," and 
Individual Adult.<£onfereuce In-School (green). Each card also contains 
categories of time usage ranging from 10 to 90 or more minutes and subfunctions 
under each main heading^ such as: attendance, behavior, and career counseling. 
Counselors can store stacks of color-coded cards *in their .desk drawers and 
complete any one of them after an interview or activity by merely' pulling one 
ouf and circling three appropriate numbers. The amount of counselor time 
devoted to this behavior is minimal/ Of course, the staff of a school using 
this system is capable of determining how often they wish to collect 
enumerative data. Copies of the cards are found in Appendix B. 

Another version of this approach is the score-sheet approach on the 
PDOE survey form found in Appendix A. This format follows the same principle. 
However, instead of using color-coded cards, a tally sheet is employed. 

To acquire opinion datar, guidance staffs will have .to systematically 
develop valid surveys for each of their important constituencies. At first, 
this will be a major task. On the othej? hand, once good surveys have been 
developed, they need only be modified as time passes. An example of a 
survey for administrator opinions about their counselors is fbhnd in appendix 

E - . ... 

Some system for-periodically surveying samples of specified constituents 
is recommended so as not to saturate them with surveys. Also, the amount of- 
counselor time' and drudgery associated with this task can be minimized 
significantly by using computer capabilities for tallying and analyzing the 
surveys. # 
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•Conclusion 

4jp^ to thi!s % pqint, I have advocated" counselor readiness for and 
acceptance of demands for accountability. Indeed, I have even suggested a~ 
approach which I deem to be both comprehensive and easy to employ. 

- ■ • ) 

In closing, L wish to comment about the attitudes which are associated 

with accountability. Whatever system is used needs to "allow for failure. 

- — * 

Thus, experimentation is encouraged. Emphasis should be placed upon 
successes, and information about failures should be used as a base for 
renewed planning for individual and programmatic* improvement. Under these 
circumstances the professional staff is less likely to fear and resist 
~efforts~tp establish an accountability plan. 

Finally, there, should accrue to counselors some sort of satisfaction 
from having evaluated themselves and their activities. One possible source 
of satisfaction is knowing that they have successfully conducted the tasks 
involved and learned the necessary skills in the process. Another source of 
satisfaction is the realization that, the acquired information can be used for 
program and personal improvement. This allows them to us'e the acquired 
information constructively. 

^ In my opinion, 'an important 'source of satisfaction would be to know 
that extrinsic rewards. are being given to those who perform above and beyond, 
the call of duty according to the data acquired in any" accountability plan. 
This would serve notice that accountability is not merely an^idea which is 
emnloyed to make individuals defend their existence but is also a means of 
rewarding the faithful. • . 



Many of the ideas I expressed herein are based upon reading .and using 
materials published by Merville' C. Shaw of California^State College at 
Chico. . ' 
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Appendix A 

The 'sample which will be used to explain the structure of the cross- 
tabulation tables is Table 21, page 23. This table shows the joint frequency 
distributions of the PEOPLE and TIME variables for the elementary level. The _ 
z T£S$mse, categories for the - PEOPLE ^variafile are listed vertically while those for 
the TIME 'variable are listed horizontally. Each cell in the table contains three 
numbers. The top, non-decimal, number is the gctual number of counseling contacts 
which were jointly described by the two response choices intersecting at that 
point in the table. The other two numbers are percentages which show the ratio of 
that top number to the row total and. column total respectively, « 

4 Perhaps the easiest way to understand the structure of the crosstabulation 
tables is to view each row and columft as a separate table describing the subtotal 
^shown in the margins. An examination of the column headed by "0-5" minutes will 
illustrate this. The margin total at the bottom of this column is 3,741'. These w 
contacts represent 15.5 percent of the total of 24,136 counseling contacts reported. J f 
The separate cells in this "0-5 minutes" column show how these 3,741 contacts were / j" 

described in reference to the PEOPLE variable. For instance, 1,512 (40.4 percent) 
of them involved "Teachers," 498 (13.3 percent) were with "Individual" students 
and 23 (0.6 percent) were with students in a "Small Group." 

An examination of the row started by "Teachers" shows- a marginal subtotal 
of 5,435 contacts. These, as shown, are 22.5 percent of the total of 24,136 
contacts reported for this level. The separate cells in this row show how these 
5,435 contacts with "Teachers" were described in terms of the response choices of 
the TIME variable. As can be seen, 1,512 (27.8 percent) were "0-5 minutes" 2,338 
(43.0 percent) were "6-15 minutes" and 56 (1.0 percent) were "61 + minutes." 
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Figure 1 
Data Collection Instrument 
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Appendix B 

Color-Qoded Data Collection Cards 
from Erie, Pennsylvania Survey 



INDIVIDUAL STUDENT COUNSELING 



SCHOOL 



COUNSELOR 



0 
1 
2 

3 
k 



Less than 
10-19 min 
20-2^ min 
30-39 min 
kO-hS. min 



10 mi n. 



5 
6 
7 
8 
9 



50-59 min. 
60-69 min. 
70-79 min. 
80-89 min. 
90 or more 



01 Attendance 

02 Behavior 

03 Career Counseling 

0k Career Decision-Making 

05 College Counseling 

06 Drop-Out 

07 Drug Counsel ing 

08 Employment Counseling 

09 Finances 

10 Home Problems 

11 Personal Social Adjustment 

12 School Program Choice 
_1_3 — Sex 

\k Subject Change 

15 Teacher-Student Problem 

16 Value Counsel ing 

17 Other 



0 Problem Identified 'f 

1 Solution Implemented 

2 Problem Solved 



INDIVIDUAL STUDENT 
INFORMATION/SERVICE 



SCHOOL 



COUNSELOR 



0 


Less than 10 min. 
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10-19 min. 
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3 
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k 
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9 


90 or 


more 



01 Attendance 
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06 Enrol 1 ing 
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08 Financial Aid, Current 

09 Free Lunch 

10 Mi 1 i tary Information 
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T3 P rog ranTTnTFb rma t i on 

]k Sick Calls 

15 Summer School 

16 Test Information 

1 7 Test I nterpretat ion 
18* Transportation 

19 Wi thdrawi ng 

20 Work Permit 
21' Othex-— 
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Appendix B 



INDIVIDUAL ADULT CONFERENCE 3 
IN-SCHOOL 
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16 Other 
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Program Choice 








15 


Other 




15 


Tardiness 










« 




16 


Testing 














17 


Test Interpretation 










Units Counted 




18 


Transportation 














19 


Other 










Units Estimated 












F 










Number of Students 








V 


| 




Served 














f • 






o 




f 
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Appendix C 
EXCERPT FROM KRUMBOLTZ'S "ACCOUNTABILITY 
MODtfL FOR COUNSELORS" ' 



ENUMERATIVE AND OUTCOME DATA 
General Goal - Decision Making: Help students learn how to make decisions wisely." 



Problem 
Identification 



178 seniors 
volunteered' 
;f o r3f oc a t to n- 
al decision- 
making program 
25- were chosen 



389 students 
complained 
that their 
initial 
schedule was 
unsatisfactory 
because of 
"Friends in 
another section 
"personality- 
conflicts with 
teacher," "jusi: 
heard of new 
requirement," 
"bored," etc. 



Method 



Group* counseling; 
vocational simu- 
lation; outside 
exploratory 
assignments 



Individual con- 
ferences; 
complete 
administrative 
procedures. 



Outcome 



19 mastered and 
applied 8-step 
vocational 
decision-making 
process, ending_ 
with written, 
tentative action 
plan, each 
verbalized how 
process could 
apply to new 
decisions; 6 
verbalized pro- 
cess but did 
not apply it to 
own vocational 
decision. 



378 schedule 
changes; effect 
on decision- 
making ability 
unknown. 



Activity 



Questionnaire 
adminis t ration 
analysis 

Preparation fo 
learning Acti- 
vities. 

Group meetings 

Job experience 
kits (reusable' 

Follow-up 
evaluation 



Conferences 
with parents 

Conferences 
with students 

Write revised 
schedule for 
computer. 

Troubleshoot 
and blow 
offsteam when 
computer sends 
faulty infor- 
mation. 



Hours. 



20 
10 

0 

25 



19. 
121 

68 



76 



Dollars 



Taken from June 1974 issue of the 
Personnel and Guidance Journal 



56 
# 

280* 
140. 

173* 

350 
999 



266 
1,694 

952 



1,064 



3,976 



19 



17 



Appendix D 

i 

• <■ SINGLE CASE STUDY 



Daily Frequency of Self-Monitored 
Positive Self-Thoughts 0 



15 
14 




Days of Observation 



TYPE A DESIGN 
OUTCOME DATA 



Taken from: Carl E. Thoresen, f, The Intensive Design: An Intimate Approach 
to Counseling Research. 11 A paper presented at the Annual Meeting of the 
American Educational Research Association* Chicago, April, 1972. 



Appendix D 
* SINGLE CASE STUDY 



t * 

€ 

Daily Frequency of Self-Mcnitored 
Positive Self-Thoughts 




\ 



TYPE AB DESIGN 
OUTCOME DATA 



\ 

x^ 

\ 



19 

Appendix E 
Counselor Effectiveness Questionnaire 



Please answer, the following questions to the best of your ability. In so 
doing, use the following format. Rate the counselor by circling the appro- 
priate (in your opinion) level of competence using the six part scale on 
the right. If you wish to r^ake any comments, use the space on the left- 
hand side of the page. 



7. level of competency in establishing 
f acilritative interpersonal relations 
with adolescents 

8. ability to conduct in-house research 
and evaluation 

9. ability to report clearly research 
and evaluation he or she has con- 
ducted 



10. 



11. 



12 



level of performance in conducting 
that part of the standardized 
testing program which is within 
his responsibility 

level of skill in conducting test 
interpretations 



ability to t assemble, organize and 
disseminate counseling information 
effectively 

13. ability to interpret information 
about pupils to parents 

14. level of effectiveness in assisting 
student to select post secondary 
educational opportunities: 

Colleges % 

Trade Schools 



Please respond in terms of each item 
beginning with the Counselor's: Low 

1. awareness of the role and function 1 
of guidance in the larger society 

2. level cf self -awareness 1 2 

3. awareness of the purpose for 1 2 
guidance in the schools 

4. apparent level of understanding of, , 1 2 
• and knowledge about, adolescence 

5. level of competency in establishing 1 2 
facilitative interpersonal relations 

with adults (e.g., faculty, admini- 
s trators , parents) 

6. ability to work cooperatively with 1 2 
- — -o ther pup H— per sonne~l~ ~spB"cl~aTt#t"s 

(e.g., fjlool psychologists, social 
workers, health personnel, coun- 
selors at other educational levels) 



Average High 
2 3 4 5 



1 2 



1 2 



1 2 



1 2 



1 2 



1 2 



1 2 



3 
3 



4 
4 



4 
4 



No oppor- 
tunity to 
Observe 

No 



No 
No 

No 

No 



No 



No 



No 



No 



No 



No 



No 



No 
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